Maximum likelihood estimation of Gaussian graphical models: Numerical implementation and topology selection

We describe algorithms for maximum likelihood estimation of Gaussian graphical models with conditional independence constraints. It is well-known that this problem can be formulated as an unconstrained convex optimization problem, and that it has a closed-form solution if the underlying graph is chordal. The focus of this paper is on numerical algorithms for large problems with non-chordal graphs. We compare different gradient-based methods (coordinate descent, conjugate gradient, and limited-memory BFGS) and show how problem structure can be exploited in each of them. A key element contributing to the efficiency of the algorithms is the use of chordal embeddings for the fast computation of gradients of the log-likelihood function. We also present a dual method suited for graphs that are nearly chordal. In this method, results from matrix completion theory are applied to reduce the number of optimization variables to the number of edges added in the chordal embedding. The paper also makes several connections between sparse matrix algorithms and the theory of normal graphical models with chordal graphs. As an extension we discuss numerical methods for topology selection in Gaussian graphical models.